146 research outputs found
Posterior Matching Scheme for Gaussian Multiple Access Channel with Feedback
Posterior matching is a method proposed by Ofer Shayevitz and Meir Feder to
design capacity achieving coding schemes for general point-to-point memoryless
channels with feedback. In this paper, we present a way to extend posterior
matching based encoding and variable rate decoding ideas for the Gaussian MAC
with feedback, referred to as time-varying posterior matching scheme, analyze
the achievable rate region and error probabilities of the extended
encoding-decoding scheme. The time-varying posterior matching scheme is a
generalization of the Shayevitz and Feder's posterior matching scheme when the
posterior distributions of the input messages given output are not fixed over
transmission time slots. It turns out that the well-known Ozarow's encoding
scheme, which obtains the capacity of two-user Gaussian channel, is a special
case of our extended posterior matching framework as the Schalkwijk-Kailath's
scheme is a special case of the point-to-point posterior matching mentioned
above. Furthermore, our designed posterior matching also obtains the
linear-feedback sum-capacity for the symmetric multiuser Gaussian MAC. Besides,
the encoding scheme in this paper is designed for the real Gaussian MAC to
obtain that performance, which is different from previous approaches where
encoding schemes are designed for the complex Gaussian MAC. More importantly,
this paper shows potential of posterior matching in designing optimal coding
schemes for multiuser channels with feedback.Comment: submitted to the IEEE Transactions on Information Theory. A shorter
version has been accepted to IEEE Information Theory Workshop 201
On the Capacity of Symmetric Gaussian Interference Channels with Feedback
In this paper, we propose a new coding scheme for symmetric Gaussian
interference channels with feedback based on the ideas of time-varying coding
schemes. The proposed scheme improves the Suh-Tse and Kramer inner bounds of
the channel capacity for the cases of weak and not very strong interference.
This improvement is more significant when the signal-to-noise ratio (SNR) is
not very high. It is shown theoretically and numerically that our coding scheme
can outperform the Kramer code. In addition, the generalized degrees-of-freedom
of our proposed coding scheme is equal to the Suh-Tse scheme in the strong
interference case. The numerical results show that our coding scheme can attain
better performance than the Suh-Tse coding scheme for all channel parameters.
Furthermore, the simplicity of the encoding/decoding algorithms is another
strong point of our proposed coding scheme compared with the Suh-Tse coding
scheme. More importantly, our results show that an optimal coding scheme for
the symmetric Gaussian interference channels with feedback can be achieved by
using only marginal posterior distributions under a better cooperation strategy
between transmitters.Comment: To appear in Proc. of IEEE International Symposium on Information
Theory (ISIT), Hong Kong, June 14-19, 201
The Reliability Function of Lossy Source-Channel Coding of Variable-Length Codes with Feedback
We consider transmission of discrete memoryless sources (DMSes) across
discrete memoryless channels (DMCs) using variable-length lossy source-channel
codes with feedback. The reliability function (optimum error exponent) is shown
to be equal to where is the rate-distortion
function of the source, is the maximum relative entropy between output
distributions of the DMC, and is the Shannon capacity of the channel. We
show that, in this setting and in this asymptotic regime, separate
source-channel coding is, in fact, optimal.Comment: Accepted to IEEE Transactions on Information Theory in Apr. 201
Fundamental limits and algorithms for sparse linear regression with sublinear sparsity
We establish exact asymptotic expressions for the normalized mutual
information and minimum mean-square-error (MMSE) of sparse linear regression in
the sub-linear sparsity regime. Our result is achieved by a generalization of
the adaptive interpolation method in Bayesian inference for linear regimes to
sub-linear ones. A modification of the well-known approximate message passing
algorithm to approach the MMSE fundamental limit is also proposed, and its
state evolution is rigorously analyzed. Our results show that the traditional
linear assumption between the signal dimension and number of observations in
the replica and adaptive interpolation methods is not necessary for sparse
signals. They also show how to modify the existing well-known AMP algorithms
for linear regimes to sub-linear ones.Comment: 45 pages, 2 figures. Under review for publication. Add some auxiliary
proof
Global Convergence Rate of Deep Equilibrium Models with General Activations
In a recent paper, Ling et al. investigated the over-parametrized Deep
Equilibrium Model (DEQ) with ReLU activation. They proved that the gradient
descent converges to a globally optimal solution at a linear convergence rate
for the quadratic loss function. This paper shows that this fact still holds
for DEQs with any general activation that has bounded first and second
derivatives. Since the new activation function is generally non-linear,
bounding the least eigenvalue of the Gram matrix of the equilibrium point is
particularly challenging. To accomplish this task, we need to create a novel
population Gram matrix and develop a new form of dual activation with Hermite
polynomial expansion.Comment: 32 pages. arXiv admin note: text overlap with arXiv:2205.13814 by
other author
- β¦